{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "0599d57c-16e4-478c-954d-89dbbd193ced",
   "metadata": {},
   "source": [
    "**LLM Workshop 2024 by Sebastian Raschka**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "14b82151-07c6-4867-b374-741258033b52",
   "metadata": {},
   "source": [
    "<br>\n",
    "<br>\n",
    "<br>\n",
    "<br>\n",
    "\n",
    "# 5) Loading pretrained weights (part 2; using LitGPT)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4d617b8f-8493-4afa-8c91-f3d1ab79795b",
   "metadata": {},
   "source": [
    "- Now, we are loading the weights using an open-source library called LitGPT\n",
    "- LitGPT is fundamentally similar to the LLM code we implemented previously, but it is much more sophisticated and supports more than 20 different LLMs (Mistral, Gemma, Llama, Phi, and more)\n",
    "\n",
    "# ⚡ LitGPT\n",
    "\n",
    "**20+ high-performance LLMs with recipes to pretrain, finetune, deploy at scale.**\n",
    "\n",
    "<pre>\n",
    "✅ From scratch implementations     ✅ No abstractions    ✅ Beginner friendly   \n",
    "✅ Flash attention                  ✅ FSDP               ✅ LoRA, QLoRA, Adapter\n",
    "✅ Reduce GPU memory (fp4/8/16/32)  ✅ 1-1000+ GPUs/TPUs  ✅ 20+ LLMs            \n",
    "</pre>\n",
    "\n",
    "## Basic usage:\n",
    "\n",
    "```\n",
    "# ligpt [action] [model]\n",
    "litgpt  download  meta-llama/Meta-Llama-3-8B-Instruct\n",
    "litgpt  chat      meta-llama/Meta-Llama-3-8B-Instruct\n",
    "litgpt  evaluate  meta-llama/Meta-Llama-3-8B-Instruct\n",
    "litgpt  finetune  meta-llama/Meta-Llama-3-8B-Instruct\n",
    "litgpt  pretrain  meta-llama/Meta-Llama-3-8B-Instruct\n",
    "litgpt  serve     meta-llama/Meta-Llama-3-8B-Instruct\n",
    "```\n",
    "\n",
    "\n",
    "- You can learn more about LitGPT in the [corresponding GitHub repository](https://github.com/Lightning-AI/litgpt), that contains many tutorials, use cases, and examples\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b1f9508e",
   "metadata": {},
   "outputs": [],
   "source": [
    "# pip install litgpt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "48cf71fa-af17-4c72-a6ab-f258a2b5a8ac",
   "metadata": {},
   "outputs": [],
   "source": [
    "from importlib.metadata import version\n",
    "\n",
    "pkgs = [\"litgpt\", \n",
    "        \"torch\",\n",
    "       ]\n",
    "for p in pkgs:\n",
    "    print(f\"{p} version: {version(p)}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fe29baa9-c3b0-493d-94b4-eaa8146d6b3c",
   "metadata": {},
   "source": [
    "- First, let's see what LLMs are supported"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0ae8df66-f391-4266-b437-a1f601a6ac40",
   "metadata": {},
   "outputs": [],
   "source": [
    "!litgpt download list"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2495037e-0068-49ad-9bed-0bcdc440727d",
   "metadata": {},
   "source": [
    "- We can then download an LLM via the following command"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fb0c202d",
   "metadata": {},
   "outputs": [],
   "source": [
    "!litgpt download microsoft/phi-2"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6caf5be0-4aa1-498f-b08a-68ff234cbea5",
   "metadata": {},
   "source": [
    "- And there's also a Python API to use the model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e057edbf",
   "metadata": {},
   "outputs": [],
   "source": [
    "from litgpt import LLM\n",
    "\n",
    "llm = LLM.load(\"microsoft/phi-2\")\n",
    "\n",
    "llm.generate(\"What do Llamas eat?\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fc775d4e",
   "metadata": {},
   "outputs": [],
   "source": [
    "result = llm.generate(\"What do Llamas eat?\", stream=True, max_new_tokens=200)\n",
    "for e in result:\n",
    "    print(e, end=\"\", flush=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "288158da",
   "metadata": {},
   "source": [
    "<br>\n",
    "<br>\n",
    "<br>\n",
    "<br>\n",
    "\n",
    "# Exercise 2: Download an LLM"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2717b67f",
   "metadata": {},
   "source": [
    "- Download and try out an LLM of your own choice (recommendation: 7B parameters or smaller)\n",
    "- We will finetune the LLM in the next notebook\n",
    "- You can also try out the `litgpt chat` command from the terminal"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
